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Abstract 


his paper reports on the proposed transfer of a paper-based 
fee proficiency exam to an online platform. We discuss 
both the potential predetermined advantages, which were the 
impetus for the project, and also some emergent benefits, which 
prompted an in-depth analysis and reconceptualisation of the exam’s 
role, which in turn we hope will promote positive washback as well 
as washforward. This change will be afforded through more granular 
feedback on student performance, which will be facilitated by the 


online platform. 
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1. Introduction 


The testing team at University of Central Lancashire (UCLan) produces a 
proficiency exam of English as a foreign language which is used in various 
situations, primarily to allow international students into programmes in their 
chosen discipline. It is known as the Test of English Language Level*\(TELL) and 
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is described as ‘English for Academic Purposes (EAP)-light’ in that it aims to 
cover the basic skill set students will draw on in their studies, albeit pitched at a 
relatively general level given that the exam can potentially be taken by a student 
of any discipline. Recently, a project was initiated to explore the viability of 
moving the exam to an online platform and we report on this in this article. This 
paper discusses the rationale for moving online and describes the test adaptation 
process and platform set up. We report here only on the receptive skills assessed 
on the UCLan-TELL, although the exam comprises all four skills. The piloting 
of the new computer-based exam is still ongoing and will be reported on at a 
later date. 


2. Initial goals of moving online 


There were a number of issues which acted as the impetus for investigating 
a new mode of delivery. The first was that, since the exam is used both in the 
UK and at partner institutions, reliability is compromised, somewhat inevitably, 
by a lack of parity of administration, which Garcia Laborda (2007) suggests 
can be addressed through computerisation. It has long been acknowledged that 
there are various ways in which the consistency may be threatened (Lado, 1961), 
including variability in exam conditions (nature of invigilation, environmental 
factors, etc.). The UCLan-TELL is a relatively high-stakes exam in that it allows 
or denies access to further study, and thus as such, threats to reliability are not 
acceptable. 


Another advantage of altering the mode of delivery was the possibility to 
improve security. There is always the possibility that exam integrity can be 
compromised, especially when operating at a distance from the UK campus, 
where it cannot be totally ensured who has access to the exam. As stated, the 
UCLan-TELL is a high-stakes exam for the candidates and consequently also 
for their tutors, as students’ success tends to impact on them. Such tutors (and 
admin staff) are consequently in a difficult position due to the stakes involved. 
Anecdotally, we understand information has on occasion been passed on, even 
if with quite benign intentions, rather than constituting any nefarious activity. 
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This may even happen at a sub-conscious level, e.g. topics covered in revision 
lessons leading up to the exam. This compromises exam integrity and, thus, 
affects exam validity, and administration-related reliability. The online format 
avoids any possibility of prior access to test content and thus eliminates any 
possibility of passing on information about the papers. 


A further driver to initiating this project was the future possibility, should the 
online version prove successful, of generating income by offering the exam 
to other institutions with similar needs, who may not have the resources to 
produce their own exam for the purpose of certifying international students’ 
readiness for higher education. With this in mind, the salience of improvements 
in key areas such as reliability and a well-argued case for its validity would be 
paramount to make it an attractive product. 


In addition, we envisage attractive features such as increased efficiency in 
scoring the objectively marked sections (reading and listening). On top of 
being inherently advantageous, this would reduce costs by obviating the need 
for human markers, or optical mark scanners and forms, as documented in 
the literature (see Chapelle & Voss, 2016). This might thus lead to a more 
marketable product, which would increase the chance of institutional project 
support. 


3. Further benefits 


In the process of researching the platforms and in having to provide a sound 
rationale for what could potentially be a costly undertaking, further advantages 
came to light. The first of these was improved authenticity. While this 
is widely-interpreted, often in terms of “how well [the test] replicates real 
life in the tasks” (Fulcher, 2010, p. 98), we tried to address authenticity in 
relation to how far the means by which candidates accessed and produced 
text were in alignment with students’ practice in general. The majority of 
students in higher education in the UK nowadays live their study lives mostly 
online (PWC, 2015); much, if not all, of their coursework is researched and 
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submitted online, and the vast majority of their reading is online (Seyenney 
& Ross, 2008), as UCLan’s Digital Shift project has widened the range and 
improved accessibility of the digital reading material. Therefore, bringing our 
English proficiency exam in line with this was felt to be a major advantage as 
it required students to read online and produce text online, given that this exam 
is most commonly used as “gatekeeping” (Davies et al., 1999, p. 66) for entry 
to higher education study in the UK. 


Another emergent benefit, while not an immediate priority, was to better manage 
individual needs of students with certain specific learning difficulties. The 
inclusivity agenda, i.e. to maximise opportunity for all learners (NCEO, 2011), 
has for too long been left on a wish-list of future improvements regarding our 
exam, but being able to operationalise this effectively and reliably contributed to 
the list of positives for an online approach. 


Above all, what became apparent through our investigation and development of 
ideas for adapting the exam and exploring possibilities in its new format was the 
potential to provide far better information to students on their performance. The 
online platform facilitates furnishing candidates with individualised information 
at a far more granular level than previously practically possible. Rather than an 
overall grade, or a grade per skill, the chosen online format allowed a breakdown 
of which sub-skills candidates demonstrated strengths or weaknesses in. For 
example, the feedback will highlight whether candidates demonstrated the 
ability to read for gist and detail, but not to infer meaning. Such information, 
combined with individualised prompts about how to improve in the weaker sub- 
skills, offers a formative element to what is usually perceived as summative 
assessment. 


4. A paradigm shift? 


In our experience, proficiency exams and other forms of summative assessment 
are viewed as an end point. They are frequently seen as an activity which needs 
to be pursued simply to allow entry to another phase of one’s life (such as study, 
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or a new job). They tend to foster a retrospective view of learning up to that 
point when the exam is taken. All in all, this approach can be summed up as 
being assessment of learning. In contrast, we felt the new candidate feedback 
format may allow a shift towards it simultaneously becoming assessment for 
learning (Gardner, 2012), seeing no good reason why the exam cannot straddle 
both functions. 


Furnished with an individual personalised score profile and associated advice 
for how to improve areas of weakness, a candidate could utilise this to inform 
future language learning. The information supplied could help not only the 
candidate, but also any tutors who may be involved in their learning by taking on 
a diagnostic role to guide future study (Shohamy, 1992). On entering the higher 
education institution, international students will need to continue developing 
their language skills (Evans, Anderson, & Eggington, 2015) and a diagnosis 
of their needs should guide them in effectively selecting from the means of 
support available to them. Thus, the exam need not be only summative, but 
also formative and inform the next stage of further language development. In 
other words, the aim is for the promotion of this information to support positive 
washback, to influence study prior to the exam (Alderson & Wall, 1993), as well 
as addressing the exam’s validity for its assigned purpose. Yet it also aims to 
enhance washforward (Andrews, Majer, Sargeant, & West, 2000), the effect of 
an exam on future learning. The format and content of high-stakes proficiency 
exams have been shown to influence aspects of the learning and teaching which 
takes place prior to that exam. Therefore, the feedback from performance on such 
exams can influence future learning and should be encouraged, e.g. addressing 
weaknesses identified by the exam outcomes. 


5. How the individualised feedback is achieved 


While the chosen platform allows for the generation of reports that give feedback 
to candidates, the default setup focusses on individual tasks. We felt that it would 
be more useful for candidates to receive feedback on their performance in the 
different skills and subskills coupled with suggestions on how to improve. For 
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example, with regards to the receptive skills, this was achieved by tagging each 
item with a description (Figure 1) depicting the underlying sub-skills (Figure 2). 
The system analyses candidates’ performances on all items that have the same 
tag and generates an automated email message detailing the scores achieved as 
well as explaining how improvements can be made on specific sub-skills and 
how this will reflect on academic language use. 


Figure 1. Item tagging 
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Our rationale for not using the default setup was that we wanted to help 
candidates think about their language ability in terms of specific skills and 
sub-skills, thus ultimately improving their understanding of what the exam is 
targeting and hence their assessment literacy. One way this could be improved 
would be to provide a visualisation of the candidate’s individual profile, in the 
form of a chart depicting their performance in the specific subskills (Figure 3). 
This would enable candidates to better understand the areas that they need to 
focus on to improve their language ability. 
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Figure 2. Subskills tags 


Figure 3. Example of candidate listening performance chart 
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6. Conclusions 


From an initial proposal to move our test to an online platform for administrative 
and business reasons arose a means to achieve potential pedagogically-focussed 
benefits. Although the project’s future is not yet secured, meaning the exam 
may not move online, the process of investigation has been of great benefit, 
prompting us to instigate improvements in the current version. 
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